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DETAILED ACTION 

Some of the IDS have been lined through because they are not available for consideration. 
Please re-submit. 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form 
the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1 ) an application for patent, published under section 122(b), by another filed in the 
United States before the invention by the applicant for patent or (2) a patent granted on an application for patent by 
another filed in the United States before the invention by the applicant for patent, except that an international 
application filed under the treaty defined in section 351 (a) shall have the effects for purposes of this subsection of an 
application filed in the United States only if the international application designated the United States and was 
published under Article 21(2) of such treaty in the English language. 

Claims 1-2, 4-5, 13-15, 17, 37, 40-41, 43-44, 52-54, 56, 76-77, 79-80, 88-90 and 92 
are rejected under 35 U.S.C. 102(e) as being anticipated by Jain et ai. [US. 6,567,980], 
As to claims 1 , 37, 40 and 76, Jain et al. disclose a computer implemented method and 
corresponding apparatus for displaying multimedia information comprising the steps/means 
for displaying a graphical user interface (GUI) on the display (column 4, lines 20-25); 
displaying, in a first area of the GUI, a representation of the multimedia information stored by 
the multimedia document (figure 2, 176), the displayed representation of the multimedia 
information comprising a representation of information of the first type and a representation of 
information of the second type (figure 2, 202); displaying a first lens covering a first portion of 
the first area (figure 2, 202); displaying, in a second area of the GUI, a representation of 
multimedia information displayed in the first portion of the first area (figure 2, 172), the 
representation of multimedia information displayed in the second area comprising a portion of 
the representation of information of the first type covered by the first lens and a portion of the 



Application/Control Number: 10/081 ,129 Page 2 

Art Unit: 2179 

As to claims 2, 41 and 77, Jain et al. also disclose displaying a first thumbnail image in the 
first area of the GUI (figure 2, 176), the first thumbnail image comprising the representation of 
information of the first type; and displaying a second thumbnail image in the first area of the 
GUI, the second thumbnail image comprising the representation of information of the second 
type (figure 2, column 4, lines 20-65). 

As to claims 4, 43 and 79, Jain et al. show determining a first time and a second time 
associating with the first lens; displaying, in the second area of the GUI, a representation of 
information of the first type occurring between the first time and the second time associated 
with the first lens; and displaying, in the second area of the GUI, a representation of 
information of the second type occurring between the first time and the second time 
associated with the first lens (figure 2, 178, column 6, line 40-60 and column 7, lines 15 
through column 8, line 60). 

As to claims 5, 44 and 80, Jain et al. also show receiving user input moving the first lens to 
cover a second portion of the first area; and responsive to the user input, automatically 
changing the information displayed in the second area of the GUI such that the 
representation of multimedia information displayed in the second area of the GUI 
corresponds to the representation of multimedia information included in the second portion of 
the first area (figure 2, receiving the user input to select the image and display in the panel 
172). 

As to claims 13, 52 and 88, Jain et al. also teach the information of the first type 
corresponding to video information (column 2, lines 6-24); and the representation of the 
information of the first type comprising one or more video keyframes extracted from the video 
information (column 2, lines 17-22 and column 4, lines 20-40). 
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As to claims 14, 53 and 89, Jain et al. provide the information of the second type 
corresponding to audio information (column 2, lines 18-20); and the representation of 
information of the second type comprises text information obtained from transcribing the 
audio information (column 2, lines 15-25). 

As to claims 15, 54 and 90, Jain et al. also provide the information of the second type 
corresponding to closed-caption (CC) text information; and the representation of information 
of the second type comprises text (column 4, lines 20-50). 

As to claims 17, 56 and 92, Jain et al. also provide receiving input indicating selection of a 
portion of the multimedia information occurring between a first time and a second time; and 
performing a first operation on the portion of the multimedia information occurring between a 
first time and a second time (column 6, lines 40-60 and column 12, lines 30-50). 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 
102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the 
subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

Claims 3, 6-7, 9-12, 18-36, 38-39, 42, 45-46, 48-51, 57-75, 78, 81-82, 84-87 and 93- 
1 1 1 rejected under 35 U.S.C. 103(a) as being unpatentable over Jain et al. [US. 6,567,980]. 
As to claims 3, 42 and 78, Jain et al. disclose displaying the portion of the representation of 
information of the first type covered by the first lens in the second area of the GUI; and 
displaying the portion of the representation of information of the second type covered by the 
first lens in the second area of the GUI (figure 2, 176 (first area)). Jain et al. fail to clearly 
teach a first panel and a second panel. However, it was well known in the art that the first and 
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second panels are displayed in the GUI area, (see figure 2, first area 172, the first portion 
contains image, the second portion contains text). It would have been obvious to one of 
ordinary skill in the art at the time of the invention to display the first and second information 
types in the first and second panels in the second GUI area in order to provide a good and 
convenient GUI. 

As to claims 6-7, 45-46 and 81-82, Jain et al. fail to clearly teach the third area of the GUI. 
However, it was well known in the art to display the third area in the GUI. It would have been 
obvious to one of ordinary skill in the art at the time of the invention to display the third area 
of the GUI in order to provide a good and convenient GUI. 

As to claims 9, 22, 48, 61 , 84 and 97, Jain et al. disclose receiving user input moving the first 
lens to cover a first portion of the first area and responsive the user input, automatically 
changing the information displayed in the second area of the GUI (figure 2, the user can 
selects any image in section 176 and automatically display in section 172). But, Jain et al. fail 
to clearly teach a third area in order to automatically change the information displayed in the 
third area of the GUI. However, it was well known in the art to display the third area in the 
GUI. It would have been obvious to one of ordinary skill in the art at the time of the invention 
to display the third area of the GUI in order to provide a good and convenient GUI. 
As to claims 10, 49 and 85, Jain et al. disclose receiving a user input moving the first lens to 
cover a second portion of the first area and responsive to the user input, automatically 
changing the information displayed in the second area of the GUI corresponds to the 
representation of multimedia information included in the second portion of the first area 
(figure 2, the first lens cover first and second portions. Together). Jain et al. fail to clearly 
teach the information displaying in the third area of the GUI. However, it was well known in 
the art to display the third area in the GUI. It would have been obvious to one of ordinary skill 
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in the art at the time of the invention to display the third area of the GUI in order to provide a 
good and convenient GUI. 

As to claim 1 1 , Jain et al. teach displaying a sub-lens covering a portion of the first area of 
the GUI corresponding to the first portion of the second area of the GUI covered by the 
second lens (figure 2, column 6, 42-51 , the sub-lens is used to select any image in the panel 
176). 

As to claim 12, Jain et al. also teach receiving a user input moving the second lens to cover a 
second portion of the second area; and responsive to the user input, automatically changing 
a position of the sub-lens to cover a portion of the first area of the GUI corresponding to the 
second portion of the second area (figure 2, column 6, 42-51 , the sub-lens is used to select 
any image in the panel 176). 

As to claims 18, 38, 57 and 93, Jain et al. disclose displaying a graphical user interface, 
displaying, in a first area of the GUI, a representation of the multimedia information stored by 
the multimedia document occurring between a start time and an end time associated with the 
multimedia document (figure 2, 178 and time line 180). Each of the video images occurs 
between a start time to an end time (column 6, lines 1-25). But, Jain et al. fail to clearly teach 
the step of displaying a first lens emphasized a portion of the first area of the GUI, the portion 
of the first area emphasized by the first lens comprising a representation of multimedia 
information occurring between a first time (t1) and a second time (t2). 
However, it was well known in the art to display a first lens emphasizing a portion of the first 
area of the GUI, the portion of the first area emphasizing by the first lens comprising a 
representation of multimedia information occurring between a first time (t1 ) and a second 
time (t2). 
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It would have been obvious to one of ordinary skill in the art at the time of the invention to 
have the portion of the first area emphasized by the first lens comprising a representation of 
multimedia information occurring between a first time (t1) and a second time (t2) in order to 
provide a good and convenient GUI. 

As to claims 19, 58 and 94, In light of rejection of the third area, Jain et al. show displaying a 
second lens emphasizing a portion of the second area of the GUI, the portion of the second 
area emphasized by the second lens comprising a representation of multimedia information 
occurring between a third time (t3) and a fourth time (t4) (each video image has different 
start-end time. So, a first image has its start-end time (t1-t2) and a second image has its 
start-end time (t3-t4). 

Jain et al. fail to clearly teach the step of displaying, in a third area of the GUI, the 
representation of multimedia information occurring between t3 and t4, the representation of 
multimedia information displayed in the third area comprising a representation of information 
of the first type occurring between t3 and t4 and a representation of information of the second 
type occurring between t3 and t4. 

However, it was well known in the art to display the third area in the GUI. It would have been 
obvious to one of ordinary skill in the art at the time of the invention to display the third area 
of the GUI in order to provide a good and convenient GUI. 

As to claims 20, 59 and 95, Jain et al. fail to clearly teach a fifth and sixth time, however, in 
light of rejection of t1 , t2, t3 and t4, it was well known in the computer art to represent of 
multimedia information occurring between a t5 and t6. It would have been obvious to one of 
ordinary skill in the art at the time of the invention to display the third area of the GUI in order 
to provide a good and convenient GUI. 
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As to claims 21 , 23, 60, 62, 96 and 98, Jain et al. fail to clearly teach the step of changing the 
position of the second lens in response to user input such that the second lens emphasizes a 
portion of the second area of the GUI comprising a representation of multimedia information 
occurring between a fifth time (t5) and a sixth time (t6). However, in light of rejection of the 
third area and t5 and t6, it would have been obvious to a person of ordinary skill in the art at 
the time of the invention to change the position of the second lens in response to user input 
such that the second lens emphasizes a portion of the second area of the GUI comprising a 
representation of multimedia information occurring between a fifth time (t5) and a sixth time 
(t6). The motivation would have been to provide a convenient and good interface. 
As to claims 24, 63 and 99, Jain et al. show the information of the first type being video 
information; the information of the second type being audio information; the representation of 
the information of the first: type comprising one or more video keyframes extracted from the 
video information; and the representation of information of the second type comprising text 
information obtained from transcribing the audio information (column 2, lines 6-24, column 2, 
lines 17-22 and column 4, lines 20-40 and column 2, lines 15-25). 

As to claims 25, 64 and 100, Jain et al. also show the information of the first type being video 
information; the information of the second type being closed-caption (CC) text information; 
the representation of the information of the first type comprising one or more video keyframes 
extracted from the video information; and the representation of the information of the second 
type comprising text information included in the CC text information (column 4, lines 20-50). 
As to claims 26, 65 and 101, Jain et al. disclose receiving information indicating a first topic; 
and analyzing the multimedia information stored in the multimedia document to identify one 
or more locations in the multimedia information that are relevant to the first topic (column 9, 
line 52 through column 10, line 21); wherein displaying the representation of the multimedia 
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information stored by the multimedia document occurring between ts and te in the first area of 
the GUI comprises highlighting the one or more locations in the multimedia information 
displayed in the first area of the GUI (column 8, lines 23-65); and wherein displaying the 
representation of multimedia information occurring between t1 and t2 in the second area of 
the GUI comprises highlighting the one or more locations in the multimedia information that 
occur between times t1 and t2 (column 6, lines 28-67). 

As to claims 27, 66 and 102, Jain et al. teach receiving input indicating selection of a portion 
of the multimedia information occurring between a selection start time and a selection end 
time; and performing a first operation on the portion of the multimedia information occurring 
between the selection start time and the selection end time (column 6, lines 40-60 and 
column 12, lines 30-50). 

As to claims 28, 39, 67 and 103, Jain et al. disclose displaying a graphical user interface, 
displaying, in a first area of the GUI, a representation of the multimedia information stored by 
the multimedia document occurring between a start time and an end time associated with the 
multimedia document (figure 2, 178 and time line 180). Each of the video images occurs 
between a start time to an end time (column 6, lines 1-25). Jain et al. also show displaying, in 
first section of a first area of the GUI, a first set of one or more video keyframes extracted 
from the video information occurring between a start time and an end time associated with 
the multimedia document (column 6, lines 42-60); in light of rejection of t1 and t2, it would 
have been obvious to a person of ordinary skill in the art at the time of the invention to display 
a first lens emphasizing a portion of the first section of the first area occurring between a first 
time and a second time and a portion of the second section of the first area occurring 
between t1 and t2, the emphasized portion of the first section of the first area comprising a 
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second set of one or more video keyframes extracted from the video information. The 
motivation would have been to provide a convenient and good interface. 
As to claims 29, 68 and 104, in light of rejection above about the displaying a second lens 
emphasizing a portion of the first section of the second area and a portion of the second 
section of the second area; and the video keyframes, it was well known in the computer art to 
have the step of "the emphasized portion of the first section of the second area comprising a 
third set of one or more video keyframes extracted from the video information occurring 
between a third time (t3) and a fourth time (t4) and the third set of the video keyframes being 
a subset of the second set of one or more video keyframes. It would have been obvious to 
one of ordinary skill in the art at the time of the invention to display the emphasized portion of 
the first section of the second area comprising a third set of one or more video keyframes 
extracted from the video information occurring between a third time (t3) and a fourth time (t4) 
in order to provide a good and convenient GUI. 

As to claims 30, 69 and 105, in light of rejection of claim 29, Jain et al. also teach outputting 
video information starting from t3 or from t4 or from a time between t3 and t4 in a first section 
of a third area of the GUI (column 6, lines 1-55 and column 13, lines 35-67); displaying text 
information corresponding to the information of the first type occurring between t3 and t4 in a 
second section of the third area of the GUI (column 14, lines 26-67). It would have been 
obvious to one of ordinary skill in the art at the time of the invention to display the 
emphasized portion of the first section of the second area comprising a third set of one or 
more video keyframes extracted from the video information occurring between a third time 
(t3) and a fourth time (t4) in order to provide a good and convenient GUI. 
As to claims 31 , 70 and 106, Jain et al. show the information of the first type being audio 
information (column 2, lines 10-25); and the text information corresponding to the information 
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of the first type being obtained from transcribing the audio information (column 4, lines 40- 
67). 

As to claims 32, 71 and 107, Jain et al also show the information of the first type being 
closed-caption text information, and the text information corresponding to the information of 
the first type being extracted from the CC text information (column 4, lines 20-60 and column 
8, line 50 through column 9, line 20). 

As to claims 33, 72 and 108, Jain et al. fail to clearly teach a first set of one or more slides 
extracting from the slides information occurring between ts and te. However, Jain et al. show 
the images instead. It was well known in the multimedia information that images can be 
displayed instead. It would have been obvious to one of ordinary skill in the art at the time of 
the invention to display the images of the GUI in order to provide a good and convenient GUI. 
As to claims 34, 73 and 109, the claim is analyzed previously discuss with respect claims 29 
and 33. 

As to claims 35, 74 and 1 10, in light of rejection above, Jain et al. fail to clearly teach the step 
of " a first set of one or more whiteboard images image extract from the whiteboard images 
information occurring between ts and te". However, the whiteboard images were well known 
in the multimedia information environment. It would have been obvious to one of ordinary skill 
in the art at the time of the invention to display the whiteboard images of the GUI in order to 
provide a good and convenient GUI. 

As to claims 36, 75 and 1 1 1 , the claim is analyzed previously discuss with respect claims 28 
and 35. 

As to claim 50, Jain et al. teach displaying a sub-lens covering a portion of the first area of 
the GUI corresponding to the first portion of the second area of the GUI covered by the 
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second lens (figure 2, column 6, 42-51, the sub-lens is used to select any image in the panel 
176). 

As to claim 51 , Jain et al. also teach receiving a user input moving the second lens to cover a 
second portion of the second area; and responsive to the user input, automatically changing 
a position of the sub-lens to cover a portion of the first area of the GUI corresponding to the 
second portion of the second area (figure 2, column 6, 42-51, the sub-lens is used to select 
any image in the panel 1 76). 

As to claim 86, Jain et al. teach displaying a sub-lens covering a portion of the first area of 
the GUI corresponding to the first portion of the second area of the GUI covered by the 
second lens (figure 2, column 6, 42-51, the sub-lens is used to select any image in the panel 

176). 

As to claim 87, Jain et al. also teach receiving a user input moving the second lens to cover a 
second portion of the second area; and responsive to the user input, automatically changing 
a position of the sub-lens to cover a portion of the first area of the GUI corresponding to the 
second portion of the second area (figure 2, column 6, 42-51 , the sub-lens is used to select 
any image in the panel 176). 

Allowable Subject Matter 

Claims 8, 16, 47, 55, 83 and 91 are objected to as being dependent upon a rejected 
base claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Mylinh Tran. The examiner can normally be reached on Mon - 
Thu from 7:00AM to 3:00PM at 571-272-4141 . 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Weilun Lo, can be reached at 571-272-4847. 

The fax phone numbers for the organization where this application or proceeding is 
assigned are as follows: 

571-273-8300 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or Public PAIR. Status information for 
unpublished applications is available through Private PAIR only. For more information about 
the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to 
the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 
(toll-free). 
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